Issues in Parallel Automatic Di erentiation

This paper shows how rst-order derivatives can be computed in parallel by considering the computational graph that underlies the evaluation of the target function. The graph can be generated eeciently from the ADOL-C computational trace and can be used to automatically deduce the structure of the Jacobian matrix and compute the Jacobian using the reverse mode of automatic diierentiation. By employing well-known graph-coloring techniques, one can dramatically decrease the number of reverse passes required. The resulting implementation performs well on the Sequent Symmetry and BBN Butterry TC2000 shared-memory multiprocessors. Lastly, we look at the problems that must be tackled to make automatic diierentiation a commonplace computing tool, and to allow for eecient implementations on high-performance computers. In our view, the key lies in nding better ways to incorporate user and/or compile-time information about the behavior of the program into the automatic diierentiation approach.